Nixfying #536

countoren · 2024-05-27T18:38:31Z

No description provided.

add commands and dev shell

it was not applied before verifying the lock

note: this commands are not fully tested. can be adjusted when needed.

afsalthaj · 2024-05-29T10:35:05Z

Thanks @countoren for nixifying 👍

I am testing your branch right now, but bumping into some errors.

nix-shell
error:
       … while calling a functor (an attribute set with a '__functor' attribute)

         at /Users/afsalthaj/github/nix/golem/default.nix:9:1:

            8| }:
            9| pkgs.rustPlatform.buildRustPackage {
             | ^
           10|   pname = "golem-cli";

       error: assertion '((cargoVendorDir == null) -> (! ((cargoSha256 == "") && (cargoHash == ""))))' failed

       at /nix/store/01sqyyi3b57wh47aax0mzyidyzg08qsk-nixpkgs-21.05pre273666.d1257435332/nixpkgs/pkgs/build-support/rust/default.nix:58:1:

           57|
           58| assert cargoVendorDir == null -> !(cargoSha256 == "" && cargoHash == "");
             | ^
           59| assert buildType == "release" || buildType == "debug";

afsalthaj · 2024-09-06T04:40:26Z

Any updates on this @countoren ? We would like to get this in, just that I was having trouble testing this branch. #536 (comment)

countoren · 2024-09-06T11:24:38Z

Hi @afsalthaj,

I missed the first comment,
did you try:

nix develop

instead of nix-shell

countoren · 2024-09-07T13:01:53Z

I think i was able to replicate the issue I will try to fix it this weekend.

countoren · 2024-09-07T13:26:48Z

i was able to nix-build on my end but I had an issue with nix build (flake)
this might mean that your local nixpkgs might refer to a set that is not compatible I will try to fix the nix build flake one
to have it on a "locked" packages as well from your side.

countoren · 2024-09-08T14:55:17Z

Just add missing explicit import
tested it here on a fresh cloned repo. test now if you can :

nix build

to get the a result with golem cli
and

nix develop

to jump into the dev shell

afsalthaj · 2024-09-09T10:34:20Z

@countoren Thank you for this. I will try nix soon and if works, we will get it merged.

afsalthaj · 2024-09-18T14:54:00Z

It hasn't worked for me. I have not been following nix for more than a year. That said, it's going to be harder for developers and maintain this, as it seems it may work in 1 machine and not in the other.


error: flake 'git+file:///..../nixed/golem' does not provide attribute 'devShells.x86_64-darwin.default', 'devShell.x86_64-darwin', 'packages.x86_64-darwin.default' or 'defaultPackage.x86_64-darwin'
       Did you mean devShells?

afsalthaj · 2024-09-18T15:03:41Z

@countoren Nix-shell worked for me

'/nix/store/ma6za5vicwdys9jr5qbfpfc3m88z0j9w-wasmtime-c2e97ff/crates/wasi-common/WASI/tools/witx/src/ast.rs' -> '/nix/store/bdsf6ihs4yqfq5zyrikf2hjf7mdvyyc1-witx-0.9.1/src/ast.rs'
'/nix/store/ma6za5vicwdys9jr5qbfpfc3m88z0j9w-wasmtime-c2e97ff/crates/wasi-common/WASI/tools/witx/src/validate.rs' -> '/nix/store/bdsf6ihs4yqfq5zyrikf2hjf7mdvyyc1-witx-0.9.1/src/validate.rs'
'/nix/store/ma6za5vicwdys9jr5qbfpfc3m88z0j9w-wasmtime-c2e97ff/crates/wasi-common/WASI/tools/witx/src/parser.rs' -> '/nix/store/bdsf6ihs4yqfq5zyrikf2hjf7mdvyyc1-witx-0.9.1/src/parser.rs'
'/nix/store/ma6za5vicwdys9jr5qbfpfc3m88z0j9w-wasmtime-c2e97ff/crates/wasi-common/WASI/tools/witx/src/phases.rs' -> '/nix/store/bdsf6ihs4yqfq5zyrikf2hjf7mdvyyc1-witx-0.9.1/src/phases.rs'
building '/nix/store/k0g60rs0b0vc68fakhbjnaz3a19cw3vl-cargo-vendor-dir.drv'...

[nix-shell:~/projects/nixed/golem]$

afsalthaj · 2024-09-18T15:05:55Z

[nix-shell:~/projects/hello/nixed/golem]$ which cargo
/nix/store/xggv43lp8l2nq5qhgpzihiwf1dsw9nl0-auditable-cargo-1.80.1/bin/cargo

Not sure rustup is still pointing to my own installation.

So in short, nix-shell finally worked for me, and not nix-develop

vigoo · 2024-09-19T18:05:39Z

Cargo.lock.nix

@@ -0,0 +1,10529 @@
+# This file is automatically @generated by Cargo.


What is generating this copy of our Cargo.lock file and what happens when we change our dependencies (which is quite frequent?)

vigoo · 2024-09-19T18:06:14Z

commands.nix

+, golem-examples ? pkgs.fetchFromGitHub { 
+            owner = "golemcloud";
+            repo = "golem-examples";
+            rev = "5165dcc7e3cbfa09f752caa96a869d284ec169aa";


Why this revision? What happens when we update golem-exampels? (we do it quite frequently)

this revision is just for "testing" this parameter will be overwritten by flake, this is used in nix-build/nix-shell only
flake suppose to be the default use (nix build) look at the comment at the flake inputs to see how to keep update.

vigoo · 2024-09-19T18:07:12Z

default.nix

+}:
+pkgs.rustPlatform.buildRustPackage {
+  pname = "golem-cli";
+  version = "0.0.98";


This is outdated now - who and when would update it? We are not keeping our version numbers anywhere else in our repo but use git tags to do releases.

it can be set to 0.0.0, I guess it will be possible to set it in a shellScript command before build from a tag but that will count as kinda of "side effect" and then you will have issue for people that will want to run:

nix build github:golemcloud/golem

without cloning it.

vigoo · 2024-09-19T18:08:22Z

default.nix

+  cargoLock = {
+    lockFileContents = builtins.readFile ./Cargo.lock.nix;
+    allowBuiltinFetchGit = false;
+     outputHashes = {


What is special about these two dependencies? What if they need to be updated? Or how do we know we need them or need something else here?

not 100% sure why nix could not include this dependencies directly from lock file, I had to include them explicitly to resolve some conflicts.
in order to update them it could be done manually if they are not updated that frequently otherwise it is possible to add another command to update-deps command in commands.nix (dev shell)

vigoo · 2024-09-19T18:11:02Z

fixOldSyntax.patch

+         )
+         .unwrap();
+
+-        println!("cargo::rerun-if-changed=build.rs");


This file tells me that this nix package is trying to build Golem with an older rust version (the :: syntax is since 1.77). We don't want to pin our codebase to an old rust version, and are potentially using new rust features so even though we could change this particular syntax back but the build can break any time if it's not using the same rust version that we are using (which is, always the latest, for now).

I can try to run an update (remove/update flake.lock) and update the dependencies we might wont need this line anymore.

vigoo · 2024-09-19T18:11:55Z

flake.nix

@@ -0,0 +1,37 @@
+{
+  inputs.golem-examples.url = "github:golemcloud/golem-examples";


Why is golem-examples special for nix, and not our other dependencies? (For example golem-wasm-rpc, golem-wasm-ast, golem-wit, etc)

I had issue building golem-examples when building golem-cli from nix
I think it was more "side effecty" setup.rs then the other projects, it was missing "Inflector" package. I was able to solve it referencing it on the nix build of golem-cli.
update-deps in the dev shell (commands.nix) will update it to latest version

vigoo · 2024-09-19T18:12:57Z

golem-cli/Cargo.toml.nix

@@ -0,0 +1,82 @@
+[package]


Why is there a copy of our golem-cli cargo file here? Who and when will keep it updated? What is the difference?

the difference between this file and Cargo.toml is the change to of golem examples to be to one that is brought by nix (inputs.golem-examples, and additional missing reference),
I think I had planned to be included in the update-deps command on the commands.nix but i might missed something I could look at it this weekend.

vigoo · 2024-09-19T18:13:30Z

nixDeps.patch

+ name = "golem-cli"
+ version = "0.0.0"
+ dependencies = [
+ "Inflector",


Why is this dependency added by a patch here?

I think I had to include it to build correctly golem-examples but I might be wrong I will check it this weekend

countoren · 2024-09-20T10:25:47Z

It hasn't worked for me. I have not been following nix for more than a year. That said, it's going to be harder for developers and maintain this, as it seems it may work in 1 machine and not in the other.
error: flake 'git+file:///..../nixed/golem' does not provide attribute 'devShells.x86_64-darwin.default', 'devShell.x86_64-darwin', 'packages.x86_64-darwin.default' or 'defaultPackage.x86_64-darwin'
       Did you mean devShells?

it was seatching for devShell did you try nix build?

marijanp · 2024-09-24T20:10:47Z

@countoren @vigoo I have been packaging golem not being aware of this branch and I have proposed some changes to upstream golem-wasm-rpc and golem-examples to allow specifying the paths through an environment variable.

The package expression can be found here NixOS/nixpkgs#344289

Having golem in nixpkgs would also not require you to maintain Nix inside of this repository

rozore added 8 commits May 26, 2024 23:05

add default.nix that builds golem-cli

fa3dcf2

add flakes to lock depenecies

c273aea

add commands and dev shell

changing mv of files to be patchPhase

157d130

it was not applied before verifying the lock

change file replacements (mv) to use patches instead

068b307

remove un used code

d477957

update commands to reflect how to update depednecies

aa22edf

note: this commands are not fully tested. can be adjusted when needed.

add missing patch creation

abe7f84

fix commands

94dd73c

vigoo requested a review from afsalthaj May 28, 2024 09:33

add missing explicit import of nixpkgs

3af0fe2

Merge branch 'main' into nixfying

e4d9366

Merge branch 'main' into nixfying

65ec319

vigoo reviewed Sep 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nixfying #536

Nixfying #536

countoren commented May 27, 2024

afsalthaj commented May 29, 2024

afsalthaj commented Sep 6, 2024

countoren commented Sep 6, 2024

countoren commented Sep 7, 2024

countoren commented Sep 7, 2024 •

edited

Loading

countoren commented Sep 8, 2024 •

edited

Loading

afsalthaj commented Sep 9, 2024

afsalthaj commented Sep 18, 2024 •

edited

Loading

afsalthaj commented Sep 18, 2024

afsalthaj commented Sep 18, 2024

vigoo Sep 19, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

vigoo Sep 19, 2024

countoren Sep 20, 2024

countoren commented Sep 20, 2024

marijanp commented Sep 24, 2024 •

edited

Loading

		@@ -0,0 +1,10529 @@
		# This file is automatically @generated by Cargo.

		@@ -0,0 +1,37 @@
		{
		inputs.golem-examples.url = "github:golemcloud/golem-examples";

Nixfying #536

Are you sure you want to change the base?

Nixfying #536

Conversation

countoren commented May 27, 2024

afsalthaj commented May 29, 2024

afsalthaj commented Sep 6, 2024

countoren commented Sep 6, 2024

countoren commented Sep 7, 2024

countoren commented Sep 7, 2024 • edited Loading

countoren commented Sep 8, 2024 • edited Loading

afsalthaj commented Sep 9, 2024

afsalthaj commented Sep 18, 2024 • edited Loading

afsalthaj commented Sep 18, 2024

afsalthaj commented Sep 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

countoren commented Sep 20, 2024

marijanp commented Sep 24, 2024 • edited Loading

countoren commented Sep 7, 2024 •

edited

Loading

countoren commented Sep 8, 2024 •

edited

Loading

afsalthaj commented Sep 18, 2024 •

edited

Loading

marijanp commented Sep 24, 2024 •

edited

Loading